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(54) Method and apparatus for generating color image mask 



(57) A subject color and a non-subject color of a 
mask is specified, and a first color vector representing 
the subject color and a second color vector representing 
the non-subject color are determined. A third color vector 
is set to be linearly independent of the first and second 
color vectors. First through third coefficients are then 
determined for the first through third color vectors. Bit 
map data representing the distribution of the first coeffi- 
cient is binarized to give binary mask data representing 
a mask. A mask area and a non-mask area are distin- 
guishably displayed in the color image. Accordingly, the 
present invention will produce a mask area whose color 
is dose to the subject color while excluding the non-sub- 
ject color. 
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Description 

BACKGI=tOUND OF THE INVENTION 
5 Reid of tlie invention 

The present Invention relates to a method of and an apparatus for generating mask data representing a mask area 
of a specific color in a color image. 

10 Descri ption of the Related Art 

in prepress process, some image processing, such as color change or sharpness enhancement, are sometimes 
required only for an area of a specific color in an original color image such as a photograph. For example, only a color 
of clothes on a model is to be changed in some cases, and sharpness enhancement is to be executed in an area other 
75 than the flesh tint of the model in other cases. When a desired processing is executed (or not executed) only for an area 
of a specific color in the original color image, a cutout mask representing the specific color area is prepared. 

Figs. 1(A) and 1(B) illustrate an original color image and a desirable mask. The original color image shown in Fig. 
1 (A) is a photograph of a model with black hair in yellow cloths with rose patterns. The rose patterns include white and 
pink petals and green leaves sun-ounding the petals. Fig. 1(B) shows a mask including a yellow area and a green area 
20 of the clothes. 

According to a conventional method of preparing a color mask by analyzing colors of a color image, colors of pixels 
in a possible mask area are analyzed to determine a primary component axis of the colors in a three dimensional color 
space. The pixels which have a greater component other than the primary component are to be excluded from the mask 
area. That is, when the color of a pixel is far from the primary component axis, the pixel is excluded from the mask area. 
25 However, it is difficult for the conventional method to distinguish a flesh tint area from a yellow area because the 
flesh tint color has a considerable fraction of yellow component. Accordingly, it is difficult to produce the mask area as 
shown in Fig. 1 (B) . which includes an yellow area while excluding a flesh tint area of Fig. 1 (A). Such problem is commonly 
observed when one of two colors having relatively close hues is specified as a mask area while the other color is to be 
excluded from the mask area. 

30 

SUWHWARY OF THE INVENTION 

An object of the invention is thus to produce a mask of a specific color while excluding another color close to the 
specific color. 

35 The present invention is directed to a method of generating mask data representing a desired color area in a color 
image. The method comprises the steps of: (a) specifying n pieces of subject colors and m pieces of non-subject colors 
for defining the desired color area, where m and n are integers: (b) obtaining n pieces of first color vectors for the n 
pieces of subject color in a color space; (c) obtaining m pieces of second color vectors for the m pieces of non-subject 
colors in the color space; (d) forming nxm combinations of the first and second color vectors, and obtaining a third color 

40 vector linearly independent of the first and second color vectors with respect to each of the nxm combinations to form 
nxm sets of bases composed of the first through third color vectors; (e) expressing color of each pixel included in the 
color image by a linear combination of the first through third color vectors with respect to each of the nxm sets of bases, 
to thereby obtain first through third coefficients for the first through third color vectors with respect to each of the nxm 
sets of bases; and (f) generating the mask data as a function of the first coefficient obtained with respect to each of the 

45 nxm sets of bases. 

The first coefficient is relatively large and the second coefficient is relatively small for an area whose color is close 
to the subject colors while the first coefficient is relatively small and the second coefficient is relatively large for another 
area whose color is close to the non-subject color. Even if the subject colors have a hue similar to those of the non- 
subject colors, mask data can be produced as a function of the first coefficient so that the mask data represents a mask 
50 area whose color is close to the subject colors while excluding another color area whose color is close to the non-subject 
colors. 

In a preferred embodiment, the step (d) comprises the steps of: transforming each of the mxn combinations of the 
first and second color vectors to fourth and fifth color vectors in a hue/saturation/brightness space; and selecting a vector 
representing white in the color space as the third color vector when saturation components of the fourth and f ifth color 
55 vectors are more than a predetermined value: and when a saturation component of at least one of the fourth and fifth 
color vectors is less than the predetennined value, rotating a hue component of one the fourth and fifth color vectors, 
whose saturation component is the greater, by a predetermined angle in the hue/saturation/brightness space to generate 
a sixth color vector and transforming the sixth color vector into the color space to generate the third color vector. 
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The step (f) comprises the step of: determining a value of the mask data for each pixel in the color image by the 
steps of: (1) determining a maximum of n pieces of first coefficients with respect to each pixel In the color image, the n 
pieces of first coefficients being obtained with respect to n pieces of bases which Include an identical one of the mpieces 
of second color vectors, to thereby obtain m pieces of the maximums for the m pieces of second color vectors; (2) 
5 determining a minimum of the m pieces of the maximums with respect to each pixel in the color image: and (3) determining 
the value of the mask data for each pixel in the color image as a function of the minimum of the m pieces of the maximums. 

The step (3) comprises the step of: binarizing the minimum with a threshold value to generate binary mask data. 

The step of binarizing comprises the steps of: preparing a histogram showing frequency of the minimum in the color 
image; and determining a local minimum of the histogram as the threshold value. 
TO In another prefered embodiment, each of the m and n is equal to one. 

The step (f) comprises the step of: binarizing the first coefficient with a threshold value to generate binary mask data. 

In still another embodiment, the method further comprises the steps of: (g) displaying a mask area represented by 
the mask data distinguishably from a non-mask area in the color image on a display device; and (i) executing the process- 
ing of the steps (a) through (g) on the mask area in the color image while specifying n' pieces of subject colors and nf 
15 pieces of non-subject colors at the step (a), to thereby generate second mask data representing a second mask area 
within the mask area. 

Each of the m, nf . n and rf is equal to one. 

In another embodiment, the step (a) comprises the steps of: (1) obtaining statistical color distribution of the color 
image; (2) determining a plurality of representative colors of the color image from the statistical color distribution, and 
20 displaying the plurality of representative colors on a display device; and (3) selecting the n pieces of subject colors and 
the m pieces of non-subject colors from the plurality of representative colors displayed on the display device. 

The above step (1 ) comprises the step of: dividing the color space into a plurality of divisional spaces; and obtaining 
a frequency of each of the plurality of divisional spaces within the color images as the statistical color distribution; and 
wherein the step (2) comprises the steps of: displaying the plurality of representative colors corresponding to at least 
25 part of the plurality of divisional spaces according to the frequency. 

The step of dividing the color space comprises the steps of: obtaining three histogram for three color components 
of the color space concerning the color image: and finding local minimums for the three histograms, and setting the local 
minimums as borders of the plurality of divisional spaces. 

In still another embodiment, the step (a) comprises the steps of: displaying a color selection image for selecting 
30 arbitrary color components on a display device; and specifying the n pieces of subject colors and the m pieces of non- 
subject colors while using the color selection image. 

The method further comprising, between the steps (d) and (e), the steps of: (1) selecting at least one of the color 
selection image and a pixel-skipped image of the color image as a subject image; (2) expressing color of each pixel 
included in the subject image by a linear combination of the first through third color vectors with respect to each of the 
35 nx/n sets of bases, to thereby obtain first through third coefficients for the first through third color vectors with respect 
to each of the rmm sets of bases; (3) generating preliminary mask data as a function of the first coefficient obtained with 
respect to each of the mm sets of bases in the step (2) for each pixel included in the subject image; and (4) displaying 
a preliminary mask area represented by the preliminary mask data distinguishably from a preliminary non-mask area 
in the subject image on the display device. 
40 The method further comprising the step of: repeatedly executing the processing of the steps (a) through (d) and ( 1 ) 
through (4) in response to a demand from a user while re-selecting the n pieces of subject colors and the m pieces of 
non-subject colors at the step (a). 

The present invention is further directed to an apparatus for generating mask data representing a desired color area 
in a color image. The apparatus comprises: means for specifying n pieces of subject colors and m pieces of non-subject 
45 colors for defining the desired color area, where m and n are integers; means for obtaining n pieces of first color vectors 
for the n pieces of subject color in a color space: means for obtaining m pieces of second color vectors for the m pieces 
of non-subject colors in the color space; means for forming nxm combinations of the first and second color vectors, and 
obtaining a third color vector linearly independent of Wie first and second color vectors with respect to each of the nxm 
combinations to form nxm sets of bases from the first through third color vectors: means for expressing color of each 
so pixel included in the color image by a linear combination of the first through tiiird color vectors with respect to each of 
the nxm sets of bases , to thereby obtain first through third coefficients for the first through third color vectors with respect 
to each of tiie nxm sets of bases; means for generating the mask data as a function of the first coefficient obtained viritii 
respect to each of the mm sets of bases. 

These and other objects, features, aspects, and advantages of the present invention will become more apparent 
55 from the following detailed description of tine prefenred embodiments witii the accompanying drawings. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

Figs. 1 (A) and 1 (B) show an original color photograph and a mask applied to the original; 

Fig. 2 is a block diagram illustrating a mask preparation system for preparing a cutout mask, embodying the invention ; 
5 Fig. 3 is a flowchart showing a procedure of preparing a cutout mask in the first en:*)odiment; 

Fig. 4 shows subject colors and non-subject colors on the Munsell's hue ring; 

Rg. 5 Is a flowchart showing details of the processing executed at step S4 in the flowchart of Fig. 3; 

Fig. 6 is a graph showing a first color vector VI and a second color vector V2: 

Rg. 7 shows the first through the third color vectors VI . V2. and V3 in an RGB space; 
10 Fig. 8 shows an HSV space; 

Fig, 9 shows a process of determining a vector other than a unit vector V^h <^ the white color as a third color vector 

V3; 

Fig. 10 is a flowchart showing a procedure of preparing a multi-tone mask; 

Fig. 1 1 is a flowchart showing a procedure of preparing a binary mask; 
15 Fig. 12 is a histogram of multi-tone mask data; 

Rg. 13 is another histogram of multi-tone mask data; 

Rg. 14 is a graph showing color areas cut out by a binary mask; 

Rg. 15 shows effects of a first mask prepared in the first embodiment; 

Figs. 1 6(A) and 1 6(B) show display of a first mask area and a second mask; 
20 Fig. 1 7 shows a subject color and non-subject colors on the Munsell's hue ring in a second embodiment; 

Figs. 18(A) and 18(B) show a first color vector V^j and a second color vector Vgj in the second embodiment; 

Fig. 19 shows first through third color vectors Vai. Vgj. and Vcij in the RGB space; 

Fig. 20 is a flowchart showing a procedure of preparing a multi-tone mask in the second embodiment; 

Fig. 21 shows a process of determining multi-tone mask data for one pixel value in the second embodiment; 
25 Rg. 22 shows the reason for determining a maximum km(i)=max{k1(i,1)} inthesecorKl embodiment; 

Fig. 23 shows the reason for applying a minimum min{kmQ)} as multi-tone mask data in the second embodiment; 

Fig, 24 is a flowchart showing a procedure of preparing a cutout mask in the third embodiment of the present 

invention; 

Fig. 25 shows division of the color space utilized in preparation of the color histogram; 
30 Fig. 26 shows an exemplified display of plural representative colors; 

Fig. 27 shows another method of dividing the color space in the third embodiment; 

Fig. 28 is a flowchart showing a procedure of preparing a cutout mask in the fourth embodiment of the present 
invention; 

Fig. 29 is a plan view illustrating a color guide 60 and a pixel-skipped image 70; 
35 Fig. 30 is a flowchart showing details of the processing executed at step S51 ; and 

Rg. 31 illustrates a subject area and a non-subject area displayed In the hue-saturation graph 60hs ard the pixel- 
skipped image 70 in the fourth embodiment. 

DESCRIPTION OF THE PREFERRED EMBODIMENT 

40 

A. First Embodiment 

Fig. 2 is a block diagram illustrating a mask preparation system for preparing a cutout masK embodying the invention. 

The mask preparation system is constructed as a computer system including a CPU 20. an original image memory 22 
45 for storing an original color image, a vector memory 24 for storing color vectors, a matrix memory 26 for storing matrix 

components as described later, a multi-tone mask memory 28 for storing multi-tone mask data, and a binary mask 

memory 30 for storing binary mask data. The system is further provided with a keyboard 32 and a mouse 34 functioning 

as input means or specification means, a scanner 36 as image input means, a color CRT 38 as display means, and a 

hard disk unit 40 as external storage means. 
so The CPU 20 implements functions of a matrix operation unit 50, a multi-tone mask operation unit 52, and a binary 

operation unit 54 by executing software programs stored in a RAM (not shown). The functions of these units will be 

described later in detail. 

Fig. 3 is a flowchart showing a procedure of preparing a cutout mask in the first embodiment. At step S1 . image 
data representing an original color image are captured by the scanner 36 and stored in the original image memory 22. 
55 The original color image is, for example, a color photograph as shown in Fig. 1(A). The original image memory 22 
includes a frame memory, and the image stored in the original image memory 22 is displayed on the color CRT 38. 

At steps S2 and S3, the user specifies a subject color and a non-subject color of the cutout process on the color 
image displayed on the color CRT 38. The subject color is a color whose area Is to be extracted as a mask, and the 
non-subject color is a color whose area is excluded from the mask. By way of example, the user specifies 'yellow', that 
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is. the background color of the clothes in Fig. 1(A). as a subject color and the "flesh tint of the model as a non-subject 
color. The user can specify the subject color and the non-subject color by pointing respective dots in the color image 
displayed on the color CRT 38 with a pointing device such as the mouse 34. Alternatively, the subject color can be 
specified by selecting plural dots of similar colors and obtaining their average color. This is also the case with the non- 
s subject color. 

The processing of steps S2 through S8 in the flowchart of Fig. 3 is repeated. The subject color and the non-subject 
color specified in the first cyde of the steps S2-S8 are specifically referred to as first-cycle subject color* and first-cycle 
non-subject color'. 

Rg. 4 shows the positions of the subject color and the non-subject color on the Munseirs hue ring. The first-cycle 
10 subject color 'yellow' and the first-cycle non-subject color flesh tint' are placed in proximity to each other on the Munseirs 
hue ring. As in this example, the present invention can specify the subject color and the non-subject color which have 
similar hues to each other. 

At step S4. the matrix operation unit 50 (Fig. 2) calculates color vectors representing the subject color and the non- 
subject color to obtain a matrix including the color vectors. 

IS Fig. 5 is a flowchart showing details of step S4. At step S1 1 . the matrix operation unit 50 normalizes a color vector 
representing the subject color to determine a first color vector. Fig. 6 is a graph showing a first color vector for the subject 
color and a second color vector for the non-subject color. A color vector representing a subject color CgLto is given by 
Vsub(R1,G1,B1), where (R1.G1,B1) respectively denote R (Red). G (Green), and B (Blue) components of the subject 
color Csuti- convenience of illustration, the G axis of the RGB space is omitted in the graph of Fig. 6. At step S1 1 . 

20 the matrix operation unit 50 determines a unit vector VI (r1 ,g1 .b1 ) of the color vector Vsub and stores the components 
(r1 ,g1.b1) of the unit vector in the vector memory 24. The unit vector denotes a vector whose length is equal to one. 
The process then goes to step S12 at which the matrix operation unit 50 determines a unit vector V2(r2.g2,b2) of a color 
vector Vnsub(R2,G2,B2) representing the non-subject color and stores the components (r2,g2,b2) of the unit vector in 
the vector memory 24 in the same manner as above. 

25 The first color vector V1 and the second color vector V2 thus obtained represent two of three vectors constituting a 
basis for expressing coloi-s in the RGB space. A unit vector independent of the first and second color vector V1 and V2 
is selected as a third color vector V3 for constructing the basis. This means that the third color vector V3 is a unit vector 
which is not a linear combination of the first and second color vectors VI and V2. For example, a unit vector Vwh 
representing 'white' can be selected as the third color vector V3. The unit vector Vwh representing white has components 



Rg. 7 shows the first through third color vectors VI, V2, and V3 in the RGB space. In this embodiment, linearly 
independent three color vectors V1 , V2, and V3 define a basis of the color space, and any color E(Re,Ge,Be) in the 
40 color space is expressed as a linear combination of the three color vectors VI . V2, and V3 as given by Equations (1 a)- 
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As described later in detail, mask data representing a mask are prepared based on the coefficient k1 for the first 
75 color vector VI regarding the subject color. If either one of the first and second color vector V1 and V2 coincides with 
the third color vector V3 representing "white", the three color vectors are not linearly Independent, and an arbitrary color 
can not be expressed by the three color vectors VI through V3 accordingly. Also, if either the first color vector V1 or the 
second color vector V2 is sufficiently dose to the third color vector V3 representing "white", the coefficient kl for the first 
color vector VI may become excessively large or small. At step SI 3 in the flowchart of Fig. 5, it is determined whether 
20 either one of the first color vector V1 and the second color vector V2 is sufficiently close to the unit white vector 

25 



When one of the color vectors V1 and V2 is sufficiently close to the unit white vector Vwh. the program proceeds to step 
S14 at which another vector other than the unit vector Vwh 'S determined as the third color vector V3. 
30 In concrete procedures of step SI 3. the first color vector V1 and the second color vector V2 are transformed to 
vectors in an HSV coordinate system or Hue/Saturation/Brightness system according to Equations (2a)-(2e) given as: 

MAX=max(r.g,b) (2a) 

35 MlN=min (r.g.b) (2b) 



H^aix:tan{ -.-(20) 
(r-g) + (r-b) 

V^MAX + MIN ..,(2d) 
2 



50 S» MAX -MIN (2e) 

In Equations (2a)-(2e). operators 'maxQ' and •minQ* respectively represent operations of selecting a maximum 
and a minimum among values in parentheses. Equations (2a)-(2e) give a hue H. a saturation S, and a brightness V for 
each of the first color vector V1 (r1 ,gl ,b1) and the second color vector V2(r2.g2.b2). 
55 Fig. 8 shows the HSV space, where the unit vector Vwh representing white exists on the coordinate axis of brightness 
V. This means that the unit white vector Vwh not have either the saturation S or the hue H. The HSV space is a 
cylindrical polar coordinate system or zre coordinate system, where the brightness V conresponds to a vertical coordinate 
z, the saturation S to a distance r from the z-axis, and the hue H to an angle 8. 
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A color vector of small saturation S is close to the unit white vector 




When at least one of the first vector V1 and the second vector V2 has the saturation S not greater than a predetermined 
level, it is determined that the color vector is sufficiently dose to the unit white vector Vwh- when each component is 
given as 8-bit data or expressed as a number in the range of 0 through 255. the predetermined level can be set equal 
to 1 0. or OAh In hexadecimal notation. 

Alternatively. It can be determined that a color vector is sufficiently close to the unit white vector Vwh when the ratio 
S/V of the saturation to the brightness Is not greater than a predetermined value. 

When at least one of the first color vector VI and the second color vector V2 Is sufficiently close to the unit white 
vector Vvi/H. another vector other than the unit white vector Vwh 'S set as the third color vector V3 at step S1 4 of Fig. 5. 
Fig. 9 shows a process of specifying another vector other than the unit white vector Vwh as the third color vector V3. 
One of the first and second color vectors V1 and V2 which is farther from the unit white vector Vwh 'S selected first. In 
the example of Fig. 9, the first color vector VI with the relatively large saturation S is selected. A value of 90 is then 
subtracted from the hue H of the selected first color vector V1 . This is equivalent to a clockwise rotation of the first color 
vector VI by 90 degrees In the HSV space. A vector (H-90.S, V) thus obtained is inversely transformed to a vector in the 
RGB coordinate system, and the R, G, and B components of the transformed vector are normalized to give a unit vector 
as the new third color vector V3. The third color vector V3 determined through such operations is also shown in Fig. 9. 
The angle of rotation In the above operation can be set equal to an arbitrary value other than 90 degrees. 

Ttie third color vector V3 thus obtained is linearly independent of the first color vector V1 and the second color vector 
V2. and therefore the three color vectors VI through V3 will constitute a basis of the color space. In other words, any 
color In the color space is expressed as a linear combination of the three color vectors VI through V3 as given by the 
above Equations (1a)-(1c). Since the third color vector V3 is obtained by rotating the hue component of the first color 
vector V1 or the second color vector V2. the third color vector V3 represents a real color. Since the three color vectors 
VI through V3 respectively represent real colors, and the coefficients k1 through k3 for these color vectors actually give 
a mixing ratio of the three real colors. 

The third color vector V3 can be set to be linearly independent of the first color vector VI and the second color 
vector V2 by a variety of methods other than the above process. For example, a cross product V1xV2 of the first and 
the second color vectors VI and V2 can be set as the third color vector V3. In this case, however, the cross product 
V1 xV2 may not represent a real color and the coefficients k1 through k3 for the respective color vectors do not accordingly 
have actual significance. It is thus preferable to determine the third color vector by rotating the hue component of the 
first color vector or the second color vector as described above. 

After the third color vector V3 is determined as above, the matrix operation unit 50 prepares a 3x3 matrix M given 
by Equations {1c) consisting of the components of the three color vectors V1 through V3 at step S15 of Fig. 5. The 3x3 
matrix M Is stored in the matrix memory 26. 

A set of coefficients K(k1 .k2.k3) for the arbitrary color E(Re.Ge.Be) is given by the following Equation (3): 

K = EM-1 (3) 

The set of coefficients K(k1 .k2,k3) can be determined by multiplying a matrix of the arbitrary color E{Re.Ge,Be) 
by an inverse of the matrix M. Therefore, at step S1 6, the matrix operation unit 50 calculates the inverse matrix and 
stores the inverse matrix M"*i in the matrix memory 26. 

After the process of step 84 shown in Fig. 5 is completed, a multi-tone mask is prepared at step S5 shown in Fig. 
3. Fig. 1 0 is a flowchart showing a procedure of preparing a multi-tone mask. At step S21 . the multi-tone mask operation 
unit 52 reads out color data E(Re,Ge,Be) of each pixel In tiie color image from the original image memory 22, and 
substitutes tiie data E(Re,Ge.Be) Into the right side of the above Equation (3) to determine the set of coefficients 
K{kl,k2.k3). The first coefficient k1 for the first color vector V1 is extracted from the set of coefficients K(k1 ,k2,k3) at 
step S22 and stored in the multi-tone mask memory 28 at step 823. 

The first coefficient k1 for each pixel represents the contribution of the first color vector VI to the color of each pixel. 
The greater the contribution of the first color vector V1 to the color of each pixel, the larger the first coefficient k1. Bit 
map data representing tiie first coefficients kl for all the pixels in the color Image will represent a multi-tone mask defining 
the area whose color is proximate to the subject color. 

When the value of the first coefficient kl is negative for a certain pixel, the pixel has a color just opposite to the 
subject color of the cut-out operation. In such cases, the value of the first coefficient k1 can be replaced by zero. 
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Alternatively, a value kk;=k1/k1a . where k1 is the first coefficient for each pixel and k1a Is the first coefficient k1a 
for the subject color (k1a=R1/r1=5G1/gl=B1/b1 ). can be used as multi-tone mask data. In this case, a value kk more 
than one can be forced to be one. and a negative value kk can be forced to be zero. This allows bit map data representing 
the multi-tone mask to be in the range of 0 through 1 . thereby simplifying the steps of setting a threshold value in bina- 
rization process described later. 

According to another structure, the three coefficients k1, k2, and k3 for each pixel are normalized according to 
Equations (4a) through (4c) given below, and bit map data of a normalized coefficient kul is used to represent a multi- 
tone mask. 
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25 The coefficient ku1 is preferably set equal to zero for the negative values. This also allows bit map data of the 

coefficient ku1 to be in the range of 0 through 1 , thereby simplifying the steps of setting a threshold value in binarization 
process described later. 

The multi-tone mask is. for example, applied to openwork composition, where the intensity of the subject color is 
utilized as a composition ratio. In another application, the strength and weight of image processing can be controlled 
30 according to the multi-tone mask. In the process of contour enhancement in an image, for example, the contour is strongly 
enhanced in portions with the high mask value and slightly enhanced in other portions with the low mask value. 

After preparation of the multi-tone mask, the binary operation unit 54 binarizes the multi-tone mask to produce a 
binary mask at step S6 of Fig. 3. A threshold value applied to the binarization process can be determined by the following 
various methods. 

35 Fig. 1 1 is a flowchart showing a procedure of preparing a binary mask, where a threshold value is input manually 
When the user inputs a threshold value through interactive operations at step S31 . the binary operation unit 54 binarizes 
the multi-tone mask with the input threshold value to produce a binary masks at step S32. The user checks the binary 
mask displayed on the color CRT 38 and determines whether the binary mask is satisfactory at steps S33 and S34. It 
is preferable to display both of the color image and the binary mask side by side as shown in Figs. 1 (A) and 1(B). When 

40 the binary mask thus prepared is unsatisfactory at step S34. the program returns to step S31 to wait for input of a new 
threshold value to execute the binarization process again. When the binary mask is satisfactory at step S34. on the 
contrary, the program exits from the binarization process. The procedure of Fig. 11 allows the user to prepare a desirable 
binary mask. 

In an alternative method, the threshold value applied to the binarization process is automatically set based on a 
45 distribution of the multi-tone mask. Fig. 12 is a histogram of multi-tone mask data. A value Kst on the abscissa of Fig. 
12 represents a multi-tone mask value for a subject color. A color image generally includes considerably many pixels 
having colors close to the subject color of cut-out operation, and a peak of the histogram accordingly appears in the 
vicinity of the mask value of the subject color. Since the background of the image has dark colors, another peak appears 
in the region of small mask values. Under such conditions, the frequency of appearance reaches a local minimum at a 
so value x smaller than the mask value Kst of the subject color as shown in Fig. 12. The mask value x. which is smaller 
than the mask value Kst of the subject color and gives a local minimum for the frequency of appearance, is set as the 
threshold value to be applied to the binarization process. The threshold value thus determined gives a binary mask 
representing a color area whose color is proximate to the subject color. 

The mask value x giving the local minimum for the frequency of appearance can be determined more easily by 
55 setting a maximum of the multi-tone mask value equal to a predetermined value Kmax and substituting all the values 
not less than Kmax by the predetermined value Kmax as shown in Fig. 13. The value Kmax is. for example. 128 when 
the multi-tone mask value is expressed by 8-bit data. 

A non-hierarchical clustering method (also called reallocation method or k-mean method) can also be applied to 
determine the threshold value of the binarization process based on the histogram of the multi-tone mask data. This 
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methcxi determines clusters In the histogram and specifies a lower limit of a cluster for the subject color as a threshold 
value, thus preparing a binary mask representing a color area close to the subject color. 

Fig. 14 is a graph showing color areas cut out by the binary mask A region (Ra+Rb) has colors whose coefficient 
k1 for the first color vector V1 is no less than a value Ta, whereas another region Rb has colors whose coefficient k1 is 
5 no less than another value Tb. A binary mask representing a color area in the region (Ra+Rb) is prepared by setting the 
threshold value of the binarization process equal to the value Ta. In this case, colors in a region Rc adjacent to the region 
Ra have smaller fractions of the subject color and are thus excluded from the binary mask. In a similar manner, a binary 
mask representing a color area in the region Rb is prepared by setting the threshold value of the binarization process 
equal to the value Tb. 

w Rg. 15 shows effects of a.flrst mask prepared by one cycle of steps S2 through S7 in the flowchart of Fig. 3. A 
number of circles in Fig. 1 5 represent a distribution of colors in the color image. The color image includes a first set Obi 
of pixels having colors close to the subject color Cgub and a second set Ob2 of pixels having colors dose to the non- 
subject color Cnsub- The first set Obi includes colors In an area glistening with reflected rays at the time of photographing 
(generally referred to as highlight area). In the original color image of Fig. 1(A). for example, the glistening area of the 

IS yellow clothes corresponds to a highlight area. According to the procedures described above, a binary mask including 
the first set Ob1 with the highlight area and excluding the second set Ob2 is prepared by setting the threshold value 
equal to a value Ta shown in Rg. 15. Since both the highlight area in the first set Ob1 and the second set Ob2 have 
large fractions of the blue component, the conventional method can not adequately separate the highlight area from the 
second set Ob2. The method of the embodiment, on the contrary, can easily distinguish the highlight area having a color 

20 proximate to the subject color from the area of the second set Ob2. 

When the color of the clothes or yellow is specified as the first-cycle subject color Csub and llesh tint' of the model 
as the first-cycle non-subject color Cnsub ^he original color image of Fig. 1 (A), a mask representing the yellow area of 
the clothes can be obtained as shown in Fig. 1(B). As clearly seen from the Munseirs hue ring of Fig. 4, 'green' has a 
hue opposite to that of the first-cycle non-subject color of flesh tint with respect to the first-cycle subject color of yellow. 

25 Specification of yellow as the first-cycle subject color Cgub and flesh tint as the first-cycle non-subject color Cpsub allows 
the green areas of the leaves to be included in the mask area accordingly This means that a binary mask including 
yellow areas and green areas is prepared as shown in Rg. 1(B). 

The first cycle of steps 82 through S8 in the flowchart of Rg. 3 gives a mask including an area whose color is dose 
to the first-cycle subject color Cs^b and exduding an area whose color is close to the first-cycle non-subject color Cnsub 

30 by simply spedfying the first-cycle subject color Csub and the first-cycle non-subject color Cpsub • 

When preparation of the first mask is completed, the binary mask data are combined with image data of the original 
color image, and a subject area of cut-out operation is displayed on the color CRT 38 at step S7 of Fig. 3. Fig. 16(A) 
shows a displayed image where a subject area and a non-subject area of cut-out operation are distinguished from each 
other. In the drawing of Fig. 16(A). the non-subject area hatched with broken lines is displayed in multi-tone gray while 

35 the first mask area, or a subject area of cut-out operation, is displayed in colors. The first mask area indudes yellow and 
green areas as shown in Fig. 1(B). Any method other than that described above can be applied to distinguish a subject 
area of cut-out operation or mask area from a non-subject area. For example, the non-subject area can be displayed in 
black and white. 

The user checks the subject area of cut-out operation displayed on the color CRT 38 and determines whether the 
40 mask obtained is satisfactory or not at step S8 of Fig. 3. When the mask is unsatisfactory the program returns to step 
82 to repeat the procedure of steps 82 through 88. In the second cycle of the procedure, a second-cycle subject color 
and a second-cycle non-subject color are specified at step 83 within the color part of the subject area of cut-out operation, 
and steps 85 through 87 are also executed on the color part of the subject area of cut-out operation. In the example 
shown In Fig. 4. the same color as the first-cycle subject color, that is, yellow Is spedfied as the second-cycle subject 
45 color, while green is specified as the second-cyde non-subject color. The first mask includes yellow and green areas 
as shown in Fig. 16(A). Selection of green as the second-cyde non-subject color within the first mask area will produce 
a second mask which only indudes a yellow area as illustrated in Fig. 16(B). When the second mask thus obtained is 
still unsatisfactory steps 82 through 88 are repeated again. The procedure of steps 82 through S8 is repeated until a 
desirable mask representing an area whose color is close to the subject color is obtained. 
so When a desirable mask is obtained, at least either one of the binary mask and the multi-tone mask is output at step 
S9. 

B. Modification of Rrst Embodiment 

55 (1) Although the RGB coordinate system is applied as the color space In the first embodiment described above, 
various color coordinate systems, such as a C1E-)CYZ coordinate system, a YMC coordinate system, or a YUV 
coordinate system can be applied instead. 

(2) The first coeffident k1 calculated according to Equation (3) or kul according to Equations (4a) can be used as 
multi-tone mask data. In this case, the value of the binary mask is set equal to one for pixels having positive multi- 
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tone mask data and equal to zero for pixels having negative multi-tone mask data. This means that the threshold 
value is set equal to zero. This alternative method gives a preferable binary mask suited for color con-ection of the 
original color image. 

(3) The first through third color vectors are not necessarily unit vectors. Each of the first through third color unit 
5 vectors, however, represents only a hue. This allows a mask area to be determined according to the hue of each 

pixel in the original color image in preparation of a mask based on the first coefficient as described previously. 

(4) Although the same color as the first-cycle subject color is independently specified as the second-cycle subject 
color in the above embodiment, it is not required to specify a subject color in the second cycle again, but the subject 
color can be assumed to be the same in all the cycles. In other cases, different colors can be specified as a subject 

10 color in respective cycles of the procedure of steps S2 through S8. 

C. Second Embodiment 

In each cycle of the procedure from steps S2 through S8 of Fig. 3. n pieces of subject colors and m pieces of non- 
75 subject colors can be specified, where n and m are integers. In this case, a mask is prepared according to the same 
procedures as the first embodiment shown in Fig. 3 with modifications described below. 

At steps S2 and S3 in the f towchart of Fig, 3. the user respectively specifies n pieces of subject colors and m pieces 
of non-subject colors of cut-out operation on the color image displayed on the color CRT 38. where at least one of n and 
m is equal to or greater than two. By way of example, the background color of the clothes, or yellow, in the drawing of 
20 Fig. 1 (A) is specified as a subject color while the color of rose leaves, or green, and flesh tint of the model are specified 
as non-subject colors. This corresponds to /t=1 and m=2. Specification of colors is executed by pointing dots in the color 
image displayed on the color CRT 38 with a pointing device, such as the mouse 34. Fig. 1 7 shows the subject color and 
the non-subject colors specified in the second embodiment. 

At step S4 of Fig. 3, the matrix operation unit 50 (Fig. 2) calculates color vectors representing the subject colors 
25 and the non-subject colors and determines matrices as described later. 

Details of the step S4 in the second embodiment are equivalent to those of the first embodiment shown in Fig. 5. 
At step Si 1 of Fig. 5, n pieces of first color vectors are calculated by normalizing n pieces of color vertors representing 
the n pieces of subject colors. Figs. 1 8(A) and 18(B) illustrate a first color vector and second color vectors, respectively. 
A color vector representing the subject color Csubl is expressed as Vsubi(RAi.GAi.BAi). where (Rai.Gai-Bai) respec- 
30 tively denote R. G. and B components of the subject color Csubi • For convenience of illustration, the G axis of the RGB 
space is omitted in the graphs of Figs. 18(A) and 18(B). At step S11, the matrix operation unit 50 determines a unit 
vector VAi(rAi.gAi.bAi) ^^e color vector Vg^bi and stores the components (rAvQAvbAi) of the unit vector In the vector 
memory 24. A unit vector denotes a vector whose length is equal to one. 

The program then goes to step SI 2 at which the matrix operation unit 50 determines unit vectors Vbi (rei .Qbi .^bi) 
35 and VB2(rB2.gB2.bB2) of color vectors VnsubiCRBi.GBi.Bei) and Vnsub2(RB2.GB2.BB2) representing the two non-subject 
colors, and stores the components (rBi.gai .^bi) and (rB2.gB2.bB2) of the unit vectors in the vector memory 24. 

Combinations of the n pieces of first color vectors Vai (i=1 through n) and the m pieces of second color vectors VBj 
G=1 through m) will make 'nxrrt sets of bases for expressing cotors in the cotor space. A unit vector independent of the 
first color vector Vai and the second color vector Vg] is selected as a third color vector Vcy to construct each basis. This 
40 means that the third color vector is a unit vector which is not defined as a linear combination of the first and second color 
vectors Va; and Vb]. The unit white vector Vwh can be selected as the third color vector Vcy. The unit white vector Vwh 
has components 



Fig. 19 shows the first through third color vectors Vai. Vb], and in the color space. In this second embodiment, 
linearly independent three color vectors Va,, Vbj. and Vcy define each set oi'nxnf bases, and any color E(Re.Ge.Be) 
in the color space is expressed as a linear combination of the three color vectors Vai. Vb], and Vci] of each set as given 
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by Equations (5a)-(5c): 
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= kl(i,j) VAi + k2(i,j) VBj + k3{i,j) Voj 
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As described later in detail, mask data representing a mask are prepared based on coefficients k1 (i j) for the first 
color vector Vai regarding the subject color. When either one of the first color vector Vyy and the second color vector Vb] 
coincides with the thiid color vector Vcn in each basis, the three color vectors are not linearly independent. Under such 
conditions, an arbitrary color can not be expressed by the three color vectors Vaj. Vbj and Vcij. When either the first color 
vector Vai or the second color vector V^j is fairly close to the third color vector Vcij, the coefficient k1 (ij) for the first color 
vector Vai may become excessively large or small. At step S13 in the flowchart of Fig. 5, it is determined whether either 
one of the first color vector Vai and the second color vector Vbj in each set is too close to the unit white vector 



VWH 



55 



When either of the color vectors Vai and Vb] is too close to the unit vector Vwh- the program proceeds to step SU at 
which a vector other than the unit white vector Vwh 'S set as the third color vector Vcij. 

in concrete procedures of step S13. the first color vector Va; and the second color vector Vb| are transformed to 
vectors in the HSV coordinate system (hue/saturation/brightness coordinate system) according to Equations (2a)-(2e) 
stated above. 

Equations (2a)-(2e) give a hue H, a saturation S. and a brightness V for each of the first color vector VAi(rAi.gAi.bAi) 
and the second color vector VBj{rBj,gBj.bB]). 

When at least one of the first color vector Vai and the second color vector Vb] Is too close to the unit white vector 
VwH. the program proceeds to step SI 4 at which a vector other than the unit white vector Vwh "S set as the third color 
vector Vcij. 

The third color vector Vcy thus obtained is linearly independent of the first color vector and the second color 
vector Vb]. so that each set of the three color vectors Vaj, Vsj. and Vcq can be used as a basis for representing colors 
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in the color space. In other words, any color in the RGB space is expressed as a linear combination of the three color 
vectors V/vj. Vgj, and Vcy as given by Equations (1a)-(1 c) stated above. The third color vector Vcy is specified by rotating 
the hue of the first color vector V^j or the second color vector Vbj as shown in Rg. 9, and the third color vector Vq^ 
accordingly represents a real color. Since the three color vectors Vaj, Vb|, and Vcy respectively represent real colors, 

5 coefficients k1{ij). k2(i,j), and k3(i,i) for these color vectors thus actually give a mixing ratio of the three real colors. 

The third color vector Vcjj can be set to be linearly independent of the first color vector V^j and the second color 
vector Vbj by a variety of methods other than the above process. For example, a cross product VAixVej of the first and 
the second color vectors VXi and Vb] can be set as the third color vector Vcij. In this case, however, the cross product 
V^iXVBj may not represent a real color and the coefficients k1 . k2. and k3 for the respective color vectors do not accord- 

10 ingly have actual significance. It is thus preferable to determine the third color vector by rotating the hue of the first color 
vector or the second color vector as described above. 

In the description below, each basisdefined by thethree color vectors Vaj. Vbj. and Vqij is referred to as Tjasis BB(i.j)". 
After the third color vector Vcy is determined as above, the program proceeds to step S15 in the flowchart of Fig. 
5. at which the matrix operation unit 50 prepares a 3x3 matrix M(i,j) according to Equation (5c). consisting of the com- 

15 ponents of the three color vectors Vaj. Vbj, and Vqi^ of each basis BB(ij. and stores the 3x3 matrix M(i.j) in the matrix 
memory 26. 

A set of coefficients K(i,j) for an arbitrary color E(Re,Ge.Be) is given by the following Equation (6): 

K(i.j) = .EM(i.i)-i (6) 

20 

The set of coefficients Kp.j) can be determined by multiplying the color data E(Re.Ge.Be) by an inverse M{i,j)"i 
of the matrix M(i,j). Therefore, at step S16. the matrix operation unit 50 calculates the inverse matrices M(i,j)"i for the 
'nxirt sets of bases BB(i,j) and stores the inverse matrices M(i.j)"^ in the matrix memory 26. 

After the process of step S4 of Fig. 3 is completed, the program proceeds to step S5 to prepare a multi-tone mask. 
25 Fig. 20 is a flowchart showing a procedure of preparing a multi-tone mask in the second embodiment. The parameters 
V and 'j' are initialized to zero at step S41 and they are respectively increased by one at steps S42 and S43. 

At step S44. the multi-lone mask operation unit 52 reads out a color data E(Re.Ge,Be) of each pixel in the color 
image from the original image memory 22, and substitutes the color data E(Re,Ge,Be) in the right side of Equation (6) 
to determine a set of coefficients K(i,1) for a basis BB{i,1). When the parameter 'i' is less than the number n at step S45, 
30 the program returns to step S43 and repeats the processing of steps S43 and S44. The repeated procedures give n 
sets of coefficients K(i.1) (i=1 through n) for the n sets of bases BB(i,1) with respect to the same second color vector 
Vbi . Fig. 21 shows a process of determining multi-tone mask data for one pixel value. The n pieces of coefficients k1(l .1) 
through k1(n,1) in the first row of Fig. 21 are obtained by the processing of steps S43 through S45 at j=1 . 

At step S46, the multi-tone mask operation unit 52 extracts the coefficients k1 (1 . 1 ) through k1 (n, 1 ) for the first color 
35 vectors V^i from the n sets of coefficients K(i.1) (i=1 through n). and determines their maximum km(i)=max{k1 (i,1)}. 

Fig. 22 shows the reason why the maximum km(i)=max{k1(i.1)} is to be obtained. The first coefficient k1(i.j) repre- 
sents the contribution of the first color vector VXi when the color E of each pixel is expressed by the basis BB(i.j). In other 
words, the greater the contribution of the first color vector Va] to the color data E of the pixel, the larger the first coefficient 
kl (i.1). In the example of Rg. 22. two coefficients k1,(1.1) and k1(2,1) are obtained with respect to one second color 
40 vector Vbi. The first color vector Vai having the greater coefficient k1(1 ,1) is closer to the color E of the pixel than the 
other first color vector Va2. When a plurality of subject colors are specified in the second embodiment, the maximum 
km{i)=max{k1(i,1)} among the n pieces of coefficients k1{l.1) (i=1 through n) with respect to the identical second color 
vector Vbi will be the one which Is obtained with the basis BB including the subject color vector closest to the color E 
of the pixel. 

45 When the maximum km(1 ) of the first coefficients is determined at j=1 . it is judged whether or not the parameter 'j' 
is less than the number m at step S47. When 'j' is less than m, the program goes to step S50 at which the parameter V 
is reset to zero and the processing of steps S42 through S47 is repeated. 

When the maximum km(j) (j=1 through m) of the first coefficients is determined for each value of the parameter j 
(j=1 through m), the program goes to step S48 at wNch a minimum value min{km(j)} among the m pieces of maximum 

so values km(j) is determined and to step S49 at which the minimum value min{km(j)} is stored as multi-tone mask data 
Knask multi-tone mask memory 28 (Fig. 21). 

Fig. 23 shows the reason for applying the minimum value min{kmG)} as the multi-tone mask data Knask- the 
example of Rg. 23. two coefficients k1 (1 , 1 ) and k1 (1 .2) are given for two color vectors Vbi and Vb2. which are respectively 
combined with one first color vector Vai . It is assumed that the coefficients kl (1 .1 ) and k1 (1 ,2) conespond to the max- 

55 imum values km(1) and km(2) obtained at step S46. As clearly shown in Rg. 23, the color E of the pixel is closer to the 
color defined by the righthand-slde second color vector Vg^ than the color defined by the lefthand-side second color 
vector Vb2. Under such conditions, the coefficient kl (1 ,1) based on the second color vector Vgi having the closer color 
becomes smaller. When there is another second color vector which is much closer to the color E of the pixel than the 
second color vector Vbi . the coefficient kl further decreases. The closer the second color vector becomes to the color 
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E of the pixel, the smaller the coefficient k1 becomes. It should be noted here that a color close to the second color 
vector for the non-subject color is to be excluded from the mask area. Selection of the minimum value among the first 
coefficients kl (1 , 1 ) and k1 (1 ,2) for the plurality of second color vectors Vbi and Vb2 as the multi-tone mask data Knask 
effectively excludes the colors which are dose to either of the plural non-subject colors from the mask area. 

5 According to the above procedure, the area whose color is dose to the n pieces of subject colors is extracted as a 
mask area while excluding the area whose color is close to the m pieces of non-subject colors from the mask area 
through determining the maximum km of the n pieces of first coefficients kl for one second color vector and selecting 
the minimum of the m pieces of maximum values km as the multi-tone mask data kmask- 

The steps in the flowchart of Fig 23 are executed for each pixel included In the color image to determine multi-tone 

w mask data k^ask for each pixel. The multi-tone mask data k^ask are bit map data representing the area whose color is 
not close to the m pieces of non-subject colors but dose to the n pieces of subject colors. 

When the multi-tone mask data k^iask negative, the corresponding pixel has a color just opposite to the subject 
color of cut-out operation. In such cases, the multi-tone mask data [^^sk may be set equal to zero. 

Alternatively, a value kka(=kmask/kmaska) obtained by dividing the multi-tone mask data k^ask each pixel by a 

15 multi-tone mask Knaska ^or the subject color (kmaskfl=f^l/r1 =G1/g1 =B1/b1) can be used as final multi-tone mask data. A 
ratio kka of more than one may be replaced by one, and a negative ratio kka can be replaced by zero. This allows bit 
map data representing the multi-tone mask to be in the range of 0 through 1 , thereby simplifying the steps of setting a 
threshold value in the binarization process described previously 

According to another modification, the three coefficients kl, k2. and k3 with respect to each pixel are normalized 

20 according to Equations (4a)-(4c) given above, and bit map data of the normalized coefficient kul is used as multi-tone 
mask data. 

A negative coefficient ku1 is preferably replaced by Zero. This also allows bit map data of the coefficient kul to be 
in the range of 0 through 1 , thereby simplifying the steps of setting a threshold value in the binarization process described 
previously 

25 Normalization according to Equations (4a)-(4c) may be executed every time when the coeffidents kl . k2. and k3 

are obtained for each basis BB(i,i) at step S44 in the ftowchart of Fig. 20. Alternatively, the multi-tone mask data Knask 

can be normalized at step S48. 

After the preparation of the multi-tone mask, the program proceeds to step S6 of Fig. 3 at which a binary mask is 

prepared based on the multi-tone mask and to step S7 at which a subject area of cut-out operation is displayed in color. 
30 When the mask obtained is unsatisfactory at step S8. the program returns to step S2 to repeat steps S2 through 

SB. This procedure is the same as the first embodiment described above. 

In the first embodiment, only flesh tint' is specified as the first-cycle non-subject color Cpsub while yellow is spedfied 

as the first-cycle subject color Csub and the green leave patterns are thus included in th e first mask (Fig. 1 (A)) . According 

to the second embodiment, on the other hand, both flesh tint' and 'green' are specified as the first non-subject colors, 
35 and the first mask includes only yellow areas accordingly Repeated execution of steps S2 through S8 in the second 

embodiment will make a mask induding only an area whose color is very close to the subject color in the color image 

made up of a large number of colors. 

As described above, the second embodiment easily provides a mask induding an area whose color is dose to n 

pieces of subject colors Caub while exduding colors close to m pieces of non-subject colors Cngub by simply specifying 
40 the n pieces of subject colors Csub and m pieces of non-subject colors Cnsub- 

D. Third Embodiment 

Fig. 24 is a flowchart showing a procedure of preparing a cutout mask in the third embodiment of the present 
45 invention. In the thiixJ embodiment, a subject color and a non-subject color are selected from typical colors induded in 
the color image. 

The procedure of Fig, 24 is the same as that of Fig. 3 except step S1a inserted between steps SI and S2. 

After image data of an original color image is captured by with the scanner 36 at step S1. a color histogram is 
prepared by re-scanning of the original color image at step S1a. A plurality of representative colors are selected from 
so the histogram and displayed on the color CRT 38. Fig. 25 shows division of the color space utilized in preparation of the 
color histogram. In the example of Fig. 25, the RGB space is partitioned by a predetermined unit length L along each 
coordinate axis to a plurality of cubic unit spaces Sp. At step S1a, a histogram of three-dimensional array Hst(i.j,k) is 
first prepared which represents the frequency of appearance of colors included in the original color image in each unit 
space Sp(i j,k), where i, j, and k denote coordinates along the R. G. and B coordinate axes and have unit coordinates 
55 equal to the unit length L. For example, when each color component of R. G, and B is expressed by 255 tone levels and 
the unit length L is equal to 5. the i, j, and k coordinates are in the range of 0 through 51 (=255/5), respectively. 

Suppose that luminance (Rn. Gn, Bn) of the color components R. G, and B of an n-th pixel in the original color image 
is (30.120.200), the corresponding i, j. and kcoordinates are respectively equal to 6(=30/5), 24(=120/5), and 40(=200/5). 
The value of the histogram Hst(6.24.40) is increased by one, accordingly Mean values Rave(i,j,k). Gave(i.j.k). and 
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Bave(i,i,k) of the respective color components in the unit space at the position of (ij.k) are simultaneously updated 
according to Equations {7a)-(7c) given below: 

^^^'^'•^•'^ H,,(iJ.h)+l • • • < 

^^-^'•i'^^ FMiliTl •••<'^> 
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Ttie above process is executed for all the pixels included in the original color image to determine the three- 
dimensional histograms Hst(i.j.k) representing the frequency of appearance of colors in each unit space and the means 
values Rave(i.i.k). Gave{i,j.k). and Bave(i,i.k) of the respective color components in the unit space. 

20 The unit length L for defining the unit space is can be set to any desirable value. When the unit length L is set equal 
to 2 to the nth power, upper several bits of the respective R, G, and B coordinates are used as the coordinates (l.j,k) of 
the histogram. This reduces the time and labor required for determining the histogram with digital computers. 

The above process may be executed not for all the pixels included in the original color image but for a pixel-skipped 
image which is obtained from the original color image. In the latter case, the re-scanning at step S1a can be replaced 

25 by skipping of inr\age data captured at step SI . 

After the histograms HstQ j,k) and the mean color components Rave, gave, and Save are obtained for the whole 
original color image, representative colors of the color Image are determined from the histogram. The representative 
colors are the colors whose frequency of appearance in the histogram Hst(i,j,k) is equal to greater than a predetennined 
threshold value. Color components of the representative colors are the mean color components (Rave, Gave. Bave) at 

30 coordinates (i.j.k). The plurality of representative colors thus determined are displayed in the order of frequency of appear- 
ance on the color CRT 38. Fig. 26 shows an exemplified display of plural representative colors, where plural color patches 
showing the representative colors are arranged on the right half of the monitor screen and a pixel-skipped image cor- 
responding to the original color image painted with the representative colors is displayed on the left half. 

In the above process of determining and displaying the representative colors, it is not necessarily required to compare 

35 the frequency of appearance of the histogram with a threshold value to determine representative colors. In this case, 
mean colors off the unit spaces will be displayed in the order of frequency of appearance. This modification also allows 
the user to easily select representative colors having the high frequency of appearance as a subject color and a non- 
subject color. 

At steps S2 and S3 of Fig. 24. the user selects a subject color and a non-subject color from the plurality of repre- 
40 sentative colors displayed on the color CRT 38. By way of example, the user specifies 'yellow', or the background color 
of the clothes shown in Fig. 1 (A), as a subject color and flesh tint' of the model as a non-subject color. In actual proce- 
dures, the user selects a color patch representing a desirable representative color among the color patches displayed 
on the color CRT 38 with a pointing device such as the mouser 34. Simultaneous display of the color patches of the 
representative colors and a color image painted wrth the representative colors as shown in Fig. 26 reduces the labor 
45 and time required for specification of the subject color and the non-subject color. Since all the representative colors are 
originally included in the color image, the presentation of the representative colors makes the user free from possible 
mistakes in specification of the subject color and the non-subject color. During the specification of the subject color and 
the non-subject color, it is not required to display a non-skipped, original color image, but display of a pixel-skipped 
image painted with the representative colors is sufficient to select subject and non-subject colors from the representative 
50 colors. 

Instead of calculating the mean color of each unit space, a color corresponding to a specific location in each unit 
space can be used as a representative color of the unit space. For example, a color coresponding to a center of each 
unit space or that corresponding to a spec'if ic corner of each unit space can be used as a representative color of the 
unit space. Calculation of mean colors in the respective unit spaces is, however, preferable since this method can deter- 
55 mine proper colors included in the original color image as representative colors. 

Other methods can be applied to divide the color space into several sections for determination of representative 
colors. Fig. 27 shows another method of dividing the color space in the third embodiment. A graph Hr on the R axis in 
Rg. 27 shows a histogram of the R component of the original color image, and a graph Hq on the G axis and a graph 
Hb on the B axis respectively denote histograms of the G and B components. As for the R component, the color space 
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is partitioned at a bottom or local minimum of the histogram Hr along the R axis into three regions Rri , Rr2. and Rr3 
in the example of Fig. 27. The color space is also partitioned at bottoms of the respective histograms Hq and Hs for G 
and B the components. When the histogram of the R component Is prepared first, for example, a histogram of the G 
component is prepared in each partition of the color space on the R axis, and a histogram of the B component is then 

5 prepared In each partition of the color space on the R and G axes. Upon condition that only one peak exists in a histogram, 
it is not required to divide the coordinate axis of the corresponding color component. Partition of the coordinate axes of 
the respective color components results in dividing the three-dimensional color space into several sections. The fre- 
quency of appearance of colors and the mean color of each space section are calculated in the same manner as 
described above. The mean color of the space section where the frequency of appearance is equal to or greater than 

10 a predetermined threshold value is specified as a representative color. 

This alternative method defines space sections applied to determination of representative colors by dividing the 
color space in the vicinity of the bottom or local minimum of the histogram of each color component. This method can 
appropriately divide the color space into several sections according to characteristics of the original color image. 
An arbitrary coordinates system of three linearly independent coordinate axes other than the R. G. and B axes can 

IS be used for the division of the color space. For example, three primary color components of the original color image can 
be used as coordinate axes for dividing the color space. 

The explanation of steps S2 through S9 of Fig. 24 are omitted here because they are executed in the same manner 
as in the first embodiment. The selection of a subject color and a non-subject color according to the third embodiment 
can be also applied to the second embodiment where a plurality of subject colors or a plurality of non-subject colors are 

20 specified. 

E. Fourth Embodiment 

Fig. 28 is a flowchart showing a procedure of preparing a cutout mask in the fourth embodiment of the present 
25 invention. In the fourth embodiment, a subject color and a non-subject color are specified using a color guide or color 
picker. 

The procedure of Rg. 28 is the same as that of Fig. 3 except steps S2 through S4 of Fig. 3 are replace by steps 
S50 through S52. 

After image data of an original color image is captured by with the scanner 36 at step S1 . a color guide and a pixel- 

30 skipped image produced from the original color image are displayed on the color CRT 38 at step S50. Fig. 29 is a plan 
view illustrating a color guide 60 and a pixel-skipped image 70 thus displayed. The color guide 60, or a color picker, is 
a color selection image for selecting a desired color. The color guide 60 shown in Fig. 29 includes a circular graph 60hs 
representing the hue and saturation and a bar graph 60^ representing the brightness. Each pixel In the hue/saturation 
graph 60hs 'S filled with a color having specific hue and saturation according to the position of the pixel. The hue is 

35 defined by the counterclockwise angle from the reference color 'red', and the saturation by the radial distance from the 
center of the circle. The brightness is expressed by the height in the brightness graph eOv^ A desired combination of 
hue. saturation, and brightness is specified by selecting one dot in the hue/saturation graph 60hs designating a 
height in the brightness graph 60v. 

Any color selection image instead of the color guide 60 shown in Fig. 29 can be used for selecting an arbitrary color 

40 to specify a subject color and a non-subject color 

At step S51 of Fig. 28, the user specifies a subject color and a non-subject color of mask preparation, and the CPU 
20 produce a mask for the color guide 60 and the pixel-skipped Image 70 accordingly. Fig. 30 is a flowchart showing 
details of the processing executed at step S51. the procedure of Fig. 30 is the same as steps S2 through S7 of Fig. 3, 
but it is directed to the preparation of a mask for the color guide 60 and the pixel-skipped image 70. 

45 At steps T1 and T2, the user specifies a subject color and a non-subject color by using the color gukie 60 displayed 
on the color CRT 38. The user can specify the subject color and the non-subject color by selecting one dot in the 
hue/saturation graph 60hs and designating a height in the brightness graph 60v both displayed on the color CRT 38 with 
a pointing device, such as the mouse 34. The dots specified in the hue/saturation graph 60hs to represent the subject 
color and the non-subject color are preferably painted with predetermined colors, for example, white and/or black. This 

so clearly shows the user the subject color and the non-subject color specified. Alternatively, the dots representing the 
subject color and the non-subject color may be shown with an identical color but in different shapes. It is also preferable 
that bars representing the brightness levels of the subject color and the non-subject color are filled with different colors. 

Since a combination of the subject color and the non-subject color is determined by repeatedly executing the process 
of steps S50 through S52 of Fig. 28, the subject color and the non-subject color specified at steps T2 and T3 of step 

55 S51 are only possible alternatives of the final subject color and non-subject color until the user gives an approval at step 
S52 as described later. The subject color and the non-subject color finally determined by the repetition of steps S50 
through S52 will be the final 'first-cycle subject color' and the final first-cycle non-subject color'. 

Steps T3 through T5 are executed in the same manner as steps S4 through S6 of Rg. 3 in the first embodiment 
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After binary masks for the color guide 60 and the pixel-skipped image 70 are prepared at step T5, the binary mask 
data are combined with image data of the hue/saturation graph 60hs and those of the pixel-skipped image 70 at step 
T6. and a subject area of cut-out operation is displayed on the color CRT 38 as a function of the combined data. Rg. 31 
illustrates a subject area and a non-subject area of cut-out operation both in the hue-saturation graph 60hs and the 
pixel-skipped image 70. In Fig. 31 . the non-subject area is hatched with broken lines and actually displayed in gray while 
the subject area is not hatched and actually displayed in color. Any method other than that described above can be 
applied to distinguish the subject area of cut-out operation, or mask area, from the non-subject area. For example, the 
non-subject area can be displayed in black and white. 

Although the subject area and the non-subject area of cut-out operation are displayed both in the hue/saturation 
graph 60hs and the pixel-skipped image 70 in Fig. 31 . the subject area may be shown only In either one of the hue/sat- 
uration graph SOhs and the pixel-skipped Image 70. In the latter case, preparation of the multi-tone mask and the binary 
mask at steps T4 and T5 of Rg. 30 are executed only for the image in which the subject area and the non-subject area 
are to be shown. 

The user checks the subject area displayed on the color CRT 38 and determines whether a desirable mask is 
obtained at step S52 of Fig. 28. When the mask prepared is unsatisfactory, the program returns to step S50 and repeats 
the processing of steps S50 and S51 . The subject color and the non-subject color finally determined by repeated exe- 
cution of steps S50 through S52 will become the final 'first-cycle subject color* and the final 'first-cycle non-subject color*. 

After the first-cycle subject color and the first-cycle non-subject color are specified, the program proceeds to step 
S5 at which a multi-tone mask is prepared for the original color image, and to step S6 at which a binary mask is prepared 
for the original color image. The matrix M obtained at step S51 for the first-cycle subject color and the first-cycle non- 
subject color is applied to the non-skipped, original color image. The processing executed at step S5 is equivalent to 
that of step T4 of Fig. 30. and the processing executed at step SB is equivalent to that of step T5. whereas only a relatively 
short time is required for the processing of steps T4 and T5 with respect to the color guide 60 and the pixel-skipped 
image 70, the processing of steps S5 and S6 for the non-skipped, original color image requires a relatively long time. 
Therefore the determination of the subject color and the non-subject color through the repetition of steps S50 through 
S52 will shorten the whole processing time, compared with repetition of steps S2 through S8 of Rg. 3 in the first embod- 
iment. 

The explanation of steps S7 through S9 of Fig. 28 are omitted here because they are executed in the same manner 
as in the first embodiment. The selection of a subject color and a non-subject color according to the fourth enrolment 
can be also applied to the second embodiment where a plurality of subject colors or a plurality of non-subject colors are 
specified. 

Although the present invention has been described and illustrated in detail, it is clearly understood that the same is 
by way of illustration and example only and is not to be taken by way of limitation, the spirit and scope of the present 
invention being limited only by the terms of the appended claims. 

Clalnds 

1 . A method of generating mask data representing a desired cotor area in a color image, said method comprising the 
steps of: 

(a) specifying n pieces of subject colors and m pieces of non-subject colors for defining said desired color area, 
where m and n are integers; 

(b) obtaining n pieces of first color vectors for said n pieces of subject color in a color space; 

(c) obtaining m pieces of second color vectors for said m pieces of non-subject colors in said color space; 

(d) forming nxm combinations of said first and second color vectors, and obtaining a third color vector linearly 
independent of the first and second color vectors with respect to each of said nxm combinations to form mm 
sets of bases composed of said first through third color vectors; 

(e) expressing color of each pixel included in said color image by a linear combination of said first through third 
color vectors with respect to each of said nxm sets of bases, to thereby obtain first through third coefficients for 
said first through third color vectors with respect to each of said rocm sets of bases; and 

(f) generating said mask data as a function of said first coefficient obtained with respect to each of said nxm 
sets of bases. 

2. A method in accordance with claim 1 . wherein said step (d) comprises the steps of: 

transforming each of said mxn combinations of said first and second color vectors to fourth and fifth color 
vectors in a hue/saturation/brightness space; and 

selecting a vector representing white in said color space as said third color vector when saturation compo- 
nents of said fourth and fifth color vectors are more than a predetermined value; and when a saturation component 
of at least one of said fourth and fifth color vectors Is less than said predetermined value, rotating a hue component 
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of one said fourth and fifth color vectors, whose saturation component is the greater, by a predetermined angle in 
said hue/saturation/brightness space to generate a sixth color vector and transforming said sixth color vector into 
said color space to generate said third color vector. 

3. A method in accordance with claim 2, wherein said step (f) comprises the step of: 

determining a value of said mask data for each pixel In said color image by the steps of: 

(1) determining a maximum of n pieces of first coefficients with respect to each pixel in said color image, said 
n pieces of first coefficients being obtained with respect to n pieces of bases which include an identical one of 
said mpieces of second color vectors, to thereby obtain mpieces of said maximums for said m pieces of second 
color vectors; 

(2) determining a minimum of said mpieces of said maximums with respect to each pixel in said color image: and 

(3) determining said value of said mask data for each pixel in said color image as a function of said minimum 
of said /n pieces of said maximums. 

4. A method in accordance with claim 3, wherein said step (3) comprises the step of: 

binarizing said minimum with a threshold value to generate binary mask data. 

5. A method in accordance with claim 4. wherein said step of binarizing comprises the steps of: 

preparing a histogram showing frequency of said minimum in said color image; and 
determining a local minimum of said histogram as said threshold value. 

6. A method in accordance with claim 2, wherein each of said m and n is equal to one. 

7. A method in accordance with claim 6, wherein said step (f) comprise the step of: 

binarizing said first coefficient with a threshold value to generate binary mask data. 

8. A method in accordance with claim 2, further comprising the steps of: 

(g) displaying a mask area represented by said mask data distinguishably from a non-mask area in said color 
image on a display device; and 

(i) executing the processing of said steps (a) through (g) on said mask area in said color image while specifying 
rf pieces of subject colors and pieces of non-subject colors at said step (a), to thereby generate second 
mask data representing a second mask area within said mask area. 

9. A method in accordance with claim 8. wherein each of said m. rrf. n and ri is equal to one. 

10. A method In accordance with claim 2. wherein said step (a) comprises the steps of: 

(1 ) obtaining statistical color distribution of said color image; 

(2) determining a plurality of representative colors of said color image from said statistical color distribution, 
and displaying said plurality of representative colors on a display device; and 

(3) selecting said n pieces of subject colors and said mpieces of non-subject colors from said plurality of rep- 
resentative colors displayed on said display device. 

11. A method in accordance with claim 10, wherein said step (1) comprises the step of: 

dividing said color space into a plurality of divisional spaces; and 

obtaining a frequency of each of said plurality of divisional spaces within said color images as said statistical 
color distribution; and wherein 

said step (2) comprises the steps of : 

displaying said plurality of representative colors corresponding to at least part of said plurality of divisional 
spaces according to said frequency. 

12. A method in accordance with claim 1 1 , wherein said step of dividing said color space comprises the steps of: 

obtaining three histogram for three color components of said color space concerning said color image: and 
f inding local minimums for said three histograms, and setting said local minimums as borders of said plurality 
of divisional spaces. 
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13. A method in accordance with claim 2, wherein said step (a) comprises the steps of: 

displaying a cotor selection image for selecting arbitrary color components on a display device: and 
specifying said n pieces of subject colors and said m pieces of non-subject colors while using said color 
selection image. 

14. A method in accordance with claim 13, further comprising, between said steps (d) and (e). the steps of: 

(1) selecting at least one of said color selection image and a pixel-skipped image of said color image as a 
sut3ject image; 

(2) expressing color of each pixel included in said subject image by a linear combination of said first through 
third color vectors with respect to each of said mm sets of bases, to thereby obtain first through third coefficients 
for said first through third color vectors with respect to each of said nxm sets of bases: 

(3) generating preliminary mask data as a function of said first coefficient obtained with respect to each of said 
nxm sets of bases in said step (2) for each pixel included in said subject image; and 

(4) displaying a preliminary mask area represented by said preliminary mask data distinguishably from a pre- 
liminary non-mask area in said subject image on said display device. 

15. A method in accordance with claim 14, further comprising the step of: 

repeatedly executing the processing of said steps (a) through (d) and (1 ) through (4) in response to a demand 
from a user while re-selecting said n pieces of subject colors and said m pieces of non-subject colors at said step (a). 

1 6. A method in accordance with claim 15, wherein each of said m and n is equal to one. 

1 7. An apparatus for generating mask data representing a desired color area in a color image, said apparatus compris- 
ing: 

means for specifying n pieces of subject colors and m pieces of non-subject colors for defining said desired 
color area, where m and n are integers: 

means for obtaining n pieces of first color vectors for said n pieces of subject color in a color space; 
means for obtaining m pieces of second color vectors for said m pieces of non-subject colors in said color 

space; 

means for forming nxm combinations of said first and second color vectors, and obtaining a third color vector 
linearly independent of the first and second color vectors with respect to each of said nxm combinations to form 
nxm sets of bases from said first through third color vectors; 

means for expressing color of each pixel included in said color image by a linear combination of said first 
through third color vectors with respect to each of said nxm sets of bases, to thereby obtain first through third 
coefficients for said first through third color vectors with respect to each of said nxm sets of bases: 

means for generating said mask data as a function of said first coefficient obtained with respect to each of 
said nxm sets of bases. 

18. An apparatus in accordance with claim 17, wherein each of said m and n is equal to one, 

19. An apparatus in accordance with claim 1 7. wherein said means for specifying colors comprises: 

means for obtaining statistical color distribution of said color image; 

means for determining a plurality of representative colors of said color image from said statistical color dis- 
tribution, and displaying said plurality of representative colors on a display device; and 

means for selecting said n pieces of subject colors and said m pieces of non-subject colors from said plurality 
of representative colors displayed on said display device. 

20. An apparatus in accordance with claim 17, wherein said means for specifying colors comprises: 

means for displaying a color selection image for selecting arbitrary color components on a display device; and 
means for specifying said n pieces of subject colors and said m pieces of non-subject colors while using said 
color selection image. 
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COLOR GUIDE 60 AND PIXEL-SKIPPED IMAGE 70 
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Fig. 30 



c 



STEP S5 1 



SPECIFY SUBJECT COLOR 
OF CUT-OUT OPERATION 



SPECIFY NON-SUBJECT COLOR 
OF CUT-OUT OPERATION 



CALCULATE COLOR VECTORS 
AND DETERMINE MATRIX 



PREPARE MULTI-TONE MASK 



PREPARE BINARY MASK 



DISPLAY SUBJECT AREA 
OF CUT-OUT OPERATION 



RETURN 



T1 
T2 
T3 

•T5 
-T6 
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g. 3 1 




IST-CYCLE NON-SUBJECT 
COLOR (FLESH TINT) 

llST-CYCLE SUBJECT 
COLOR (YELLOW) 



BRIGHTNESS 



BLUE 




60hs 60v 

^ V ' 

60 



ONLY MASK AREA IS DISPLAYED IN COLOR. 
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